Results on principal component filter banks: colored noise suppression and existence issues

We have made explicit the precise connection between the optimization of orthonormal filter banks (FBs) and the principal component property: the principal component filter bank (PCFB) is optimal whenever the minimization objective is a concave function of the subband variances of the FB. This explains PCFB optimality for compression, progressive transmission, and various hitherto unnoticed white-noise, suppression applications such as subband Wiener filtering. The present work examines the nature of the FB optimization problems for such schemes when PCFBs do not exist. Using the geometry of the optimization search spaces, we explain exactly why these problems are usually analytically intractable. We show the relation between compaction filter design (i.e., variance maximization) and optimum FBs. A sequential maximization of subband variances produces a PCFB if one exists, but is otherwise suboptimal for several concave objectives. We then study PCFB optimality for colored noise suppression. Unlike the case when the noise is white, here the minimization objective is a function of both the signal and the noise subband variances. We show that for the transform coder class, if a common signal and noise PCFB (KLT) exists, it is, optimal for a large class of concave objectives. Common PCFBs for general FB classes have a considerably more restricted optimality, as we show using the class of unconstrained orthonormal FBs. For this class, we also show how to find an optimum FB when the signal and noise spectra are both piecewise constant with all discontinuities at rational multiples of π

Filterbank optimization with convex objectives and the optimality of principal component forms

This paper proposes a general framework for the optimization of orthonormal filterbanks (FBs) for given input statistics. This includes as special cases, many previous results on FB optimization for compression. It also solves problems that have not been considered thus far. FB optimization for coding gain maximization (for compression applications) has been well studied before. The optimum FB has been known to satisfy the principal component property, i.e., it minimizes the mean-square error caused by reconstruction after dropping the P weakest (lowest variance) subbands for any P. We point out a much stronger connection between this property and the optimality of the FB. The main result is that a principal component FB (PCFB) is optimum whenever the minimization objective is a concave function of the subband variances produced by the FB. This result has its grounding in majorization and convex function theory and, in particular, explains the optimality of PCFBs for compression. We use the result to show various other optimality properties of PCFBs, especially for noise-suppression applications. Suppose the FB input is a signal corrupted by additive white noise, the desired output is the pure signal, and the subbands of the FB are processed to minimize the output noise. If each subband processor is a zeroth-order Wiener filter for its input, we can show that the expected mean square value of the output noise is a concave function of the subband signal variances. Hence, a PCFB is optimum in the sense of minimizing this mean square error. The above-mentioned concavity of the error and, hence, PCFB optimality, continues to hold even with certain other subband processors such as subband hard thresholds and constant multipliers, although these are not of serious practical interest. We prove that certain extensions of this PCFB optimality result to cases where the input noise is colored, and the FB optimization is over a larger class that includes biorthogonal FBs. We also show that PCFBs do not exist for the classes of DFT and cosine-modulated FBs

Role of principal component filter banks in noise reduction

The purpose of this paper is to demonstrate the optimality properties of principal component filter-banks for various noise reduction schemes. Optimization of filter-banks (FB's) for coding gain maximization has been carried out in the literature, and the optimized solutions have been observed to satisfy the principal component property, which has independently been studied. Here we show a strong connection between the optimality and the principal component property; which allows us to optimize FB's for many other objectives. Thus, we consider the noise-reduction scheme where a noisy signal is analyzed using a FB and the subband signals are processed either using a hard-threshold operation or a zeroth order Wiener filter. For these situations, we show that a principal FB is again optimal in the sense of minimizing the expected mean-square error

Discrete multitone modulation with principal component filter banks

Discrete multitone (DMT) modulation is an attractive method for communication over a nonflat channel with possibly colored noise. The uniform discrete Fourier transform (DFT) filter bank and cosine modulated filter bank have in the past been used in this system because of low complexity. We show in this paper that principal component filter banks (PCFB) which are known to be optimal for data compression and denoising applications, are also optimal for a number of criteria in DMT modulation communication. For example, the PCFB of the effective channel noise power spectrum (noise psd weighted by the inverse of the channel gain) is optimal for DMT modulation in the sense of maximizing bit rate for fixed power and error probabilities. We also establish an optimality property of the PCFB when scalar prefilters and postfilters are used around the channel. The difference between the PCFB and a traditional filter bank such as the brickwall filter bank or DFT filter bank is significant for effective power spectra which depart considerably from monotonicity. The twisted pair channel with its bridged taps, next and fext noises, and AM interference, therefore appears to be a good candidate for the application of a PCFB. This is demonstrated with the help of numerical results for the case of the ADSL channel

Role of principal component filter banks in noise reduction

The purpose of this paper is to demonstrate the optimality properties of principal component filter-banks for various noise reduction schemes. Optimization of filter-banks (FB's) for coding gain maximization has been carried out in the literature, and the optimized solutions have been observed to satisfy the principal component property, which has independently been studied. Here we show a strong connection between the optimality and the principal component property; which allows us to optimize FB's for many other objectives. Thus, we consider the noise-reduction scheme where a noisy signal is analyzed using a FB and the subband signals are processed either using a hard-threshold operation or a zeroth order Wiener filter. For these situations, we show that a principal FB is again optimal in the sense of minimizing the expected mean-square error

Optimizing the capacity of orthogonal and biorthogonal DMT channels

The uniform DFT filter bank has been used routinely in discrete multitone modulation (DMT) systems because of implementation efficiency. It has recently been shown that principal component filter banks (PCFB) which are known to be optimal for data compression and denoising applications, are also optimal for a number of criteria in DMT communication. In this paper we show that such filter banks are optimal even when scalar prefilters and postfilters are used around the channel. We show that the theoretically optimum scalar prefilter is the half-whitening solution, well known in data compression theory. We conclude with the observation that the PCFB continues to be optimal for the maximization of theoretical capacity as well

Nonuniform principal component filter banks: definitions, existence, and optimality

The optimality of principal component filter banks (PCFBs) for data compression has been observed in many works to varying extents. Recent work by the authors has made explicit the precise connection between the optimality of uniform orthonormal filter banks (FBs) and the principal component property: The PCFB is optimal whenever the minimization objective is a concave function of the subband variances of the FB. This gives a unified explanation of PCFB optimality for compression, denoising and progressive transmission. However not much is known for the case when the optimization is over a class of nonuniform Fbs. In this paper we first define the notion of a PCFB for a class of nonuniform orthonormal Fbs. We then show how it generalizes the uniform PCFBs by being optimal for a certain family of concave objectives. Lastly, we show that existence of nonuniform PCFBs could imply severe restrictions on the input power spectrum. For example, for the class of unconstrained orthonormal nonuniform Fbs with any given set of decimators that are not all equal, there is no PCFB if the input spectrum is strictly monotone

Filter Bank Optimization with Applications in Noise Suppression and Communications

A filter bank (FB) is used to analyze or decompose a signal into several frequency bands, which are processed separately and then combined. This allows us to allocate processing resources in a manner tailored to the distribution of the relevant signal features among the bands. A judicious allocation leads to improved system performance over direct processing of the input signal (without using an FB). FBs have found applications in almost every area of modern digital signal processing, including audio, image and video compression and communications. The main thrust of this thesis is towards the optimization of FBs based on the statistical properties of their input. We establish the optimality of a type of FB called the principal component filter bank (PCFB) for numerous signal processing problems. The PCFB depends on the input power spectrum and on the class of M channel orthonormal FBs over which we seek to optimize the FB. PCFB optimality for compression and progressive transmission has been observed to varying degrees in the past. Our work provides a unified framework for orthonormal FB optimization, that includes these earlier results as special cases. It also covers many other problems not observed earlier, notably in noise suppression and communications. A central result that we establish is that the PCFB is the optimum orthonormal FB whenever the minimization objective is a concave function of the vector of subband variances of the FB. Many signal processing problems result in such objectives. The earlier results on PCFB optimality for compression can be explained by this framework. Another example not noticed earlier is FBbased white noise reduction using zeroth order Wiener filters or hard thresholds in the subbands. Yet another case involves the discrete multitone modulation (DMT) communication system, used in ADSL (asymmetric digital subscriber line) and wireless OFDM (orthogonal frequency division multiplexing) technologies. These systems use the transmultiplexer configuration of an FB, which is usually chosen as a DFT or cosine-modulated FB for efficiency of implementation. We show that at increased implementation cost, we can minimize the transmission power requirement (for a given bitrate and error probability) by using the PCFB associated with a certain normalized noise spectrum. We present simulation examples with realistic channel and noise models for the ADSL system to compare the performance of the PCFB against other types of FBs, such as the DFT. We study various extensions of the basic PCFB optimality result. The noise suppression problem becomes more involved when the noise is colored, because the objective then depends on both signal and noise subband variances. For a specific FB class, namely the orthogonal transform coder class, we show that a simultaneous PCFB for the signal and noise is optimal (if it exists). For the class of unconstrained FBs, this does not hold in general; we develop an algorithm that computes the best FB for piecewise constant spectra. In some cases, PCFB optimality extends to classes of biorthogonal FBs too, although there are many open problems in this area, as we point out. We study the effect of nonexistence of a PCFB on the FB optimizations and show how they usually become analytically intractable. We show that PCFBs do not exist for the classes of DFT and cosine-modulated FBs. We also study nonuniform FB optimization: We establish the definition of nonuniform PCFBs and study their existence and optimality, which are shown to be much more restricted when compared with uniform PCFBs. Lastly, we study a related open problem on the parameterization of nonuniform perfect reconstruction (PR) FBs of various classes, such as the rational and FIR classes. Not all nonuniform PRFBs can be built by the common method of using tree structures of uniform PRFBs. Given a set of decimators, is there a rational PRFB using them? If so, what are all the PRFBs possible? When are they necessarily derivable from a tree structure? Very little is known about the answers to many such questions. For example, for existence of rational PRFBs with a given set of decimators, certain conditions on the decimators are known to be necessary, while certain others are sufficient. However, conditions that are both necessary and sufficient are unknown. One of our contributions is to strengthen considerably the known conditions. This is an important step towards a complete PR theory for nonuniform filter banks.</p